Stateful encoders + documentation refresh - #66
Merged
Conversation
Completes the stateful AbstractEncoder pair begun with LevelEncoder:
- RandomProjection{HV, T}: fixed D×d projection matrix (gaussian /
bipolar / sparse_ternary), per-type nonlinearities, scalar-or-vector
ternary threshold θ with a data-driven target_sparsity constructor,
supplied-matrix constructor, rethreshold sharing R, and
nearest-neighbour-only decode (lossy encoding: no analytic inverse).
- phase_encode(z, β): single shared FHRR phase-encoding helper; both
LevelEncoder's fractional power path and RandomProjection's FHRR
nonlinearity (random Fourier features) route through it.
- Regression tests for two known sharp edges: the ternary constructor's
positional collision (supplied matrix vs training data; the residual
square-matrix ambiguity and silently ignored kwargs are recorded as
TODO §2.6, fix is a follow-up rename) and BipolarHV's raw-bits /
zero-state polarity traps, pinned as behaviour-locking tests so the
obvious-but-wrong idioms trip red instead of shipping corruption.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolves the src/encoding.jl conflict by dropping the level/encodelevel/ decodelevel/convertlevel family (deleted by the encoders work) while keeping this branch's docstring reformatting for every surviving combinator. Fallout fixed: - api.md listed the four removed functions in an @docs block; restructured into Combinators / Encoders sections matching the primitives -> combinators -> encoders taxonomy, and added decode, AbstractEncoder, LevelEncoder, RandomProjection and rethreshold. - iris.jl used all four removed functions, and also defined local helpers named encode and decode which now collide with the package's exported ones. Ported to a single LevelEncoder and renamed the helpers to encodeflower/decodeflower. The prose now names the concept it was already explaining by hand: a stateful encoder is shared state, which is what makes encode and decode consistent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keeps this branch's heading hierarchy and refined prose, and takes two things from the other draft: the HDC-concept/kitchen-analogy table in the opening, which orients the reader before any code, and the closing wrap-up, which this version lacked entirely. Also updates the 'more encoders' admonition, which still advertised the removed level encoders, to name the combinator/encoder split and the new encoders. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New Literate tutorial giving the encoder layer its first documentation: the token path, the KMer/NGram distinction (with a worked equivalence to the combinators each is built from), Sequence/BagOfSymbols, and short sections on LevelEncoder and RandomProjection pointing at the dedicated tutorials. The headline example is language recognition from character k-mers, ported from the HDC workshop: seven Wikipedia snippets, one encode call each, and the similarity ranking recovers real linguistic structure (English/Simple English, Dutch/West-Flemish, then the Romance and Germanic pairs) with cross-family pairs at the bottom. This is the payoff for issue #53 and the clearest demonstration of why KMer is its own strategy. Also documents the extension point with a working custom strategy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Gives RandomProjection its first tutorial coverage, built on the workshop's colour example: - encoding RGB triples so perceptual closeness becomes hypervector similarity - blending by bundling - decode as clean-up against a codebook, with the reason it cannot be inversion - an associative memory learning each category's average colour via bind/bundle/unbind, and the same thing again with two-thirds of every observation replaced by noise (multiple-instance learning) - the tuning knobs: matrix flavours, theta, and rethreshold sharing the matrix - the kernel connection: FHRR + gaussian is random Fourier features, verified numerically against exp(-b^2 d^2/2) in the page itself - a plain-Julia perceptron sharpening the yellow/green boundary (0.85 -> 0.96), explicitly flagged as not-yet-package-API Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
index.md pointed at the wrong repository and its @contents block referenced a non-existent examples.md, so the manual section rendered empty. Replaced with a real landing page: what HDC is, installation, a runnable first taste, the three-layer taxonomy as a table, and links to every tutorial. Also adds the developer guide to the nav (it was built but unreachable) and passes repo as Remotes.GitHub, which silences Documenter's 'unable to determine repository root' navbar warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second act for the iris example: the same nearest-prototype workflow, but with the four measurements encoded by a single RandomProjection instead of the key-value + level construction. Scores comparably (0.96 vs 0.93) from an encoder that needs no domain knowledge, which sets up the real trade-off: the key-value representation can be decoded feature by feature, a random projection cannot. Includes a measured warning about standardising. The usual advice is to standardise before projecting, but on iris that makes things markedly worse (0.83 +/- 0.06 vs 0.98 +/- 0.03 raw, raw winning 29 of 30 splits) because the features already share a unit and the largest-spread ones are the most discriminative. Softened the same blanket advice in CLAUDE.md to match what the data actually shows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Literate passes '#hide' through to the markdown, but Documenter still renders the value of the block's last expression, so Random.seed!(42) was printing a stray Random.TaskLocalRNG(). Terminated with a semicolon and a hidden nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ts chance level Two small API additions, both prompted by questions that turned out to have teeth. 1. Bundling is m-way, not pairwise. For BinaryHV, BipolarHV, RealHV and FHRR the bundling rule depends on how many inputs there are, so bundling is not associative. Chained 'x + y + z' was already safe (Julia parses a chain of + as one variadic call, which reaches bundle with all three), but 'sum(hvs)', 'reduce(+, hvs)' and '(x + y) + z' fold pairwise and bundle a bundle. Measured on five hypervectors, the folded result scores 0.08/0.05/0.14/0.26/0.50 against its inputs where a true bundle scores ~0.37 for all five: the defining property of bundling is lost. 'sum' now bundles the whole collection; the remaining pairwise spellings are documented as wrong in the bundle docstring. 2. A similarity score is uninterpretable without knowing the metric: 0.35 means 'unrelated' under Jaccard (BinaryHV, GradedHV) and 'clearly related' under cosine. Adds similaritymetric(HV) and, more usefully, chancesimilarity(HV) — the score of an unrelated pair, which is the number you actually need. Both are locked by a test asserting the documented baseline matches what random hypervectors really score, and the encoding tutorial now teaches reading a similarity against its chance level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Runic-formats the two new tutorials; every other changed .jl was already conformant, so the style CI should be clean. - Aligns the colours tutorial's advice on standardising with what the iris tutorial actually measures (it is not an automatic win). - Stops tracking docs/src/examples/*.md. These are Literate output regenerated by docs/make.jl on every build, and they accounted for a third of this branch's diff (1431 of 4383 lines) as machine-generated duplicates of the .jl sources. Verified the docs build from a clean state with no .md present. - Gitignores .claude/ (local dev config). - Syncs CLAUDE.md and TODO.md: the new similarity traits, the m-way bundling rule, the tutorial list, and a stale entry still advertising the removed level/encodelevel/decodelevel/convertlevel family. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 27, 2026
Closed
Closed
Collaborator
|
I'll review this one as it supersedes everything else; give me the afternoon to do this properly, as it's a big change driven by Claude. |
Collaborator
Author
|
We can discuss it later today |
Audited the documentation I drafted using the ai-writing-tells skill, with the
project's own prose (the tutorials I did not write) as the style baseline.
Layers 0-1 were clean: no chatbot artifacts, no citation problems. Era-tagged
vocabulary: no hits anywhere. Copula density sits inside the project's own range
(1.2-3.4 per 100 words), so there was no copulative avoidance to undo.
Fixed:
- index.md opened with a four-item list of virtues ('fast, robust to noise,
remarkably data-efficient, and simple enough...') plus an unsupported
comparison to deep learning. Replaced with the concrete reason the paradigm is
robust and fast, and attributed the broader claim to the review paper.
- 'That is the entire/whole X' had become a verbal tic, appearing four times
across three files. Reworded two of them.
- Literal em dashes: the project uses '--' and never a literal em dash, in any
file, by any author. My additions had introduced them, all space-surrounded.
Converted in index.md, inference.jl and my section of encode.jl. The stateful
encoder docstrings that arrived with the encoders branch are left alone.
- Trimmed two rhetorical intensifiers in colours.jl.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mechanical only: keyword-argument spacing, comprehension indentation, and one trailing space. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
|
This looks good; I'll merge it and close the other PRs superseded by this one. Should we publish this as v1.0, or do you want to add something else before that? |
Collaborator
|
This PR supersedes multiple PRs and issues. This comment indicates the effect of this PR:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the stateful encoders and the tutorial rewrite together, ports the tutorials to the
current API, and gives the encoder layer its first documentation.
The two strands are combined because they are not separable: the encoder work deletes
level/encodelevel/decodelevel/convertlevel, and the tutorials use all four. Mergingeither side alone leaves
mainwith docs that do not build.Replaces
encoders→ mainencoders→docs-new-tutorials38-integration38-integration→ main38-integrationplus the encoder mergecvigilv:docs-new-tutorials#28 (
docs_types) still touchessrc/vectors.jl, which no longer exists — it looks obsolete,but nothing here deliberately covers it, so that is a separate call.
What is in it
Stateful encoders (from #64):
LevelEncoderfor scalars andRandomProjectionfor featurevectors, both under
AbstractEncoder{HV}, withencode/decodeandrethreshold. Thisreplaces the old
level/encodelevel/decodelevel/convertlevelfamily, and along the wayretires the bug where
convertlevelbuilt its encoder and decoder over two different randomladders.
Tutorials brought to the current API:
iris.jlused all four removed functions, and defined local helpers namedencodeanddecodethat now collide with the package's exports. Ported to a singleLevelEncoder;helpers renamed. The prose got simpler: it used to explain by hand that encoder and decoder
must share a ladder, which is exactly what
LevelEncodernow enforces by construction.api.mdlisted the four removed functions in an@docsblock (a hard build error).Restructured into Combinators / Encoders matching the layer taxonomy, which also gives
the stateful encoders their first API-page entry.
the concept/kitchen-analogy table and the closing wrap-up from docs: new tutorials #38.
New documentation:
KMervsNGramdistinction (each shown equivalent to the combinators it is built from),
Sequence/BagOfSymbols, and the extension point. Headline example is language recognition fromcharacter k-mers: seven Wikipedia snippets, one
encodecall each, and the similarity rankingrecovers real linguistic structure (English/Simple-English, Dutch/West-Flemish, then the
Romance and Germanic pairs) with cross-family pairs at the bottom. This is the payoff for New k-mers encoder #53.
RandomProjectionend to end: encoding, blending,decodeas codebook clean-up(and why it cannot be inversion), an associative memory recovering each category's average
colour, the same with two-thirds of every observation replaced by noise, the tuning knobs, and
a plain-Julia perceptron sharpening a boundary (0.85 → 0.96). The kernel connection is
verified numerically in the page: FHRR + Gaussian matches
exp(-β²d²/2)to three decimals.RandomProjectioncall, scoring comparably(0.96 vs 0.93), which sets up the real trade-off between an interpretable record encoding and
a one-shot projection.
@contentsreferenced anon-existent
examples.md, so the manual section rendered empty); developer guide added to thenav;
Remotes.GitHubsilences the navbar warning.Two small API additions, both from review questions on the docs:
sum(hvs)now bundles the whole collection. Bundling is m-way and therefore notassociative for
BinaryHV/BipolarHV/RealHV/FHRR. Chainedx + y + zwas already correct(Julia parses a
+chain as one variadic call), butsum,reduce(+, …)and(x + y) + zfold pairwise and bundle a bundle. Measured on five hypervectors, the folded result scores
0.08/0.05/0.14/0.26/0.50against its inputs where a true bundle scores ~0.37for all five —the defining property of bundling is silently lost.
sumnow does the right thing; theremaining pairwise spellings are documented as wrong.
similaritymetric(HV)andchancesimilarity(HV). A similarity score is uninterpretablewithout its metric:
0.35means unrelated under Jaccard (BinaryHV,GradedHV) andclearly related under cosine.
chancesimilarityreturns the score of an unrelated pair(
0.0cosine,1/3Jaccard) and is locked by a test asserting the documented baseline matcheswhat random hypervectors actually score.
Breaking
level,encodelevel,decodelevel,convertlevelare removed — useLevelEncoder.sum(hvs)now bundles all at once instead of folding pairwise (the old result was wrong).Notes for review
docs/src/examples/*.mdis Literate output regenerated on every build; it was tracked and madeup a third of this diff as machine-generated duplicates of the
.jlsources. Now untracked andgitignored — verified the docs build from a clean state with no
.mdpresent.features before a random projection hurts on iris (0.83 ± 0.06 vs 0.98 ± 0.03 raw, raw
winning 29 of 30 splits) because its features already share a unit and the largest-spread ones
are the most discriminative; and the pairwise-bundling degradation above.
Verification
866 tests, doctests, and a warning-clean
docs/make.jlbuild all green. Every changed.jlchecked against Runic, so the style job should be clean.